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Wc report on an improved measurement of the Cabibbo-Kobayashi-Maskawa CP-violating phase 7 
through a Dahtz plot analysis of neutral D meson decays to K^ti^tx^ and K%K^ K~ produced in 
the processes B"^ DK^, B^ D*K^ with D* Dtv°,Dj, and B"^ DK*^ with K*^ 
K^TT^ . Using a sample of 383 million BB pairs collected by the BABMi detector, we measure 
7 = (76±22±5±5)° (mod 180°), where the first error is statistical, the second is the experimental 
systematic uncertainty and the third reflects the uncertainty on the description of the Dalitz plot 
distributions. The corresponding two standard deviation region is 29° < 7 < 122°. This result has 
a significance of direct CP violation (7 7^ 0) of 3.0 standard deviations. 
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I. INTRODUCTION AND OVERVIEW 

In the Standard Model (SM) the phase in the Cabibbo- 
Kobayashi-Maskawa (CKM) quark-mixing matrix [1] is 
the sole source of CP violation in the quark sector of 
the electroweak interactions. This phase can be directly 
determined using a variety of methods, involving either 
the interference between decays with and without mixing 
in time-dependent CP asymmetries in neutral B meson 
decays, or interference between neutral B (self-tagged) 
or charged B decays yielding the same final state (di- 
rect CP violation). These multiple determinations in 
CP- violating tree-level processes as well as in decays in- 
volving penguin diagrams test the CKM mechanism, thus 
probing the presence of physics beyond the SM [2] . 
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FIG. 1: Main Feynman diagrams contributing to the B — > 
D°K- decay. The left dia gram proceeds via b — » cms transi- 
tion, while the right diagram proceeds via b — > ucs transition 
and is color suppressed. 

Among these determinations, the measurement of the 
angle 7, defined as arg[-V^^V*i,/V^^V*i,], where Vij are 
the elements of the CKM matrix, is one of the most dif- 
ficult to achieve and constitutes an important goal of 
present and future B physics experiments. Several meth- 
ods have been proposed to extract 7. However, those 
using ST i)(*)o^(*)T decays [3, 4] (the symbol 
indicates either a or a D^*)^ meson) are theoreti- 

cally clean and are unlikely to be affected by new physics 
because the main contributions to the amplitudes come 
from tree-level diagrams, as shown in Fig. 1. This is an 
important distinction from most of other direct measure- 
ments of phases of CKM elements. The decay amplitudes 
for the color allowed B- d(*)^K^*)'- {b cus) and 
the color suppressed B^ jj{*)Oj^{*)- (5 _^ ^^g-j tran- 

T7 



sitions [5] differ by a factor r^''e'^*i 



Here, r^' is 



the magnitude of the ratio of the amplitudes A{B 
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D^*^^K~) and A{B- -> D'^*^°K-) and is their rel- 
ative strong phase. The weak phase 7 leads to different 
B~ and B^ decay rates (direct CP violation) and, when 
the and decay to a common final state [6~ 

9], the phases become observable. The uncertainty in 7 
scales roughly as l/r^g. From the ratio of CKM matrix 

elements we expect r^' « cp \ KsKl I / I ^us^cb I be 
approximately in the range 0.1 — 0.2, where cp ~ 0.2 — 0.4 
is the color suppression factor [10, 11]. 

When the neutral D meson is reconstructed in a 
three-body final state, like K'^tt'^tt^, the distribution 
in the Dalitz plot [12] depends on the interference be- 
tween Cabibbo allowed, doubly-Cabibbo suppressed, and 
CP-eigenstate decay amplitudes of D° (from B^ 
£)(*)o^(*)T) and D° (from B^ |)(*)o^(*)t). The 
dominant interfering amplitudes in the K'^tt^tt~ final 
state are D° K*-Tr+, D° ~> K*+n-, and -> 
K^p{770f [5, 13]. Neglecting effects from D° - D° 
mixing and CP asymmetries in neutral D decays that 
are 1% or less [14-16], the B^ &*^°K'^, with 
f)*o _^ Z)07r°,i)°7, £)" K°TT+TT- decay chain ampli- 



tude A^:^\m'^, mj^) can be written as 

A'C\ml,ml) oc Adt + Xr's^ e'^-^B' Ti) 



D± 



(1) 



where and are the squared invariant masses of 
the K'^tt^ and K'^tt^ combinations, respectively, and 
Ad^ = -4d('7i^, mj_), with Ad- i-^D+) the amplitude 
of the D" KItt+tt- {W -> KIt:-t:+) decay For con- 
venience, TOq is defined analogously for the tt+tt^ com- 
bination. The factor A in Eq. (1) takes the value —1 for 
the decay B^ — > D*^[D'^"i\K^ and +1 for the remaining 
B decays. This relative sign arises due to charge conju- 
gation and angular momentum conservation in the D*^ 
decay [17]. The corresponding decay rate r^''(r7i^, to^) 
can therefore be written as 
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„(*) 



} , (2) 



where we introduce the CP parameters [18] x"^^ 
(*) 



cos{Sg^ =F 7) and y'^' = r^^' sui{Sg' =F 7), where 



„(*) 



(*), 



,(*) 



Decays ^ D^K*^ with K*^ /sTOtt^ [4] are 
also used in this analysis. For these, Eq. (2) requires the 



replacements 



J*) 

ST' 'S 
(*) 



6s, X 



Xszp = KTs cos(i5s T 7)' and yy ys^ = nr^ sin((5s =F l), 



where 



'ST 



where [11] 
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Here, Ac{p) and arc the magnitudes of the b ^ c 

and b u amplitudes as a function of the — > 
D^KgTT^ phase space position p, and 6{p) is the relative 
strong phase. The parameter k accounts for the interfer- 
ence between B=F D^K*"^ and other D^Klir'^ 
amplitudes with < k < 1 in the most general case. 
This effective parameterization also accounts for effi- 
ciency variations as a function of the kinematics of the 
B decay. 



In this paper we present an improved measurement of 
7 based on the analysis of the Dalitz plot distribution of 
£|0 ^ K^s'^+'K- and, for the first time, £>° K°K+K-, 
using a sample of 351 fb~"^ of integrated luminosity 
recorded at the Y{4S) resonance, corresponding to 383 
million BB pairs. We analyze a total of seven sig- 
nal samples (also referred to as CP samples), B~ — > 
D'-*^°K- and B- D^K*', with D*° £)°7r°,£»°7, 
£>° ^ K^iT+7r-,K°K+K-, K*- K^n' [5]. Due 
to lack of statistics, the decay B~ — > D^K*~ with 
K^K^K^ has been excluded from the analysis. 
We also reconstruct high statistics control samples, one 
for each signal B decay channel: B^ D^*^^t:~ and (for 
B- D°K*~) B~ ~> D°a^ with aj" tt-tt+tt- [5, 19]. 
We exploit the same dataset to determine Ad^ for 
Kg7r~^TT~ and Z>° K^K^K~ decays from analyses of 
the respective Dalitz plots for high-statistics samples of 
flavor-tagged mesons from £>*+ D^-k'^ decays [5], 
produced in e+e~ — > cc events. Additional improve- 
ments compared to our previous publication [18] include 
a higher reconstruction efficiency, an optimized treat- 
ment of the e+e^ ^ iq, Q = d, s, c background and 
an improved i^^Tr+Tr^ description of the Dalitz 

plot distribution (referred to hereafter as Dalitz model), 
resulting in significant decrease of statistical, systematic 
and model uncertainties. This measurement supersedes 
our previous result based on 227 million BB pairs [18]. 



The paper is organized as follows. In Sec. II we de- 
scribe the reconstruction and selection of the signal and 
control samples. Sec. Ill is devoted to the determination 
of Adt for £»° ^ K^n+n- and D° K^K+R- de- 
cays. In Sec. IV we describe the simultaneous maximum 
likelihood fit to the distributions and F^' for the 

B^ D^*^^K^ samples and to the analogous distribu- 
tions for B^ — » D^K*^ , to determine the CP parameters 
, 2/^ , Xg^^ and j/s^p- In that section we also present 
the experimental results, including systematic uncertain- 
ties. We extract these CP parameters since they have a 
good Gaussian behavior for small values of r^^Krg and 
relatively low statistics samples, independent of their val- 
ues and precisions, in contrast to 7, r^'', J^'', /trg, and 6s- 
Finally, in Sec. V, we interpret the experimental results 
and extract the physically relevant quantities 7, r ^ , Og , 
KTs and 6s, using a statistical (frequentist) analysis. 



II. EVENT SELECTION 

A. BABAR detector 

This analysis is based on a data sample collected by the 
BaBar detector at the Stanford Linear Accelerator Cen- 
ter PEP-II e+e" asymmetric-energy storage ring. The 
BaBAR detector is described in detail elsewhere [20] . We 
summarize briefly the components that are crucial to this 
analysis. Charged-particle tracking is provided by a five- 
layer silicon vertex tracker (SVT) and a 40-layer drift 
chamber (DCH). In addition to providing precise space 
coordinates for tracking, the SVT and DCH also mea- 
sure the specific ionization (dE/dx), which is used for 
particle identification of low-momentum charged parti- 
cles. At higher momenta {p > 0.7 GeV/c) pions and 
kaons are identified by Cherenkov radiation detected in 
a ring-imaging device (DIRC). The typical separation be- 
tween pions and kaons varies from 8a at 2 GeV/c to 2.5a 
at 4 GeV/c, where a denotes here the standard deviation. 
The position and energy of photons are measured with 
an electromagnetic calorimeter (EMC) consisting of 6580 
thallium-doped Csl crystals. These systems are mounted 
inside a 1.5-T solenoidal super-conducting magnet. We 
use a GEANT4-based Monte Carlo (MC) simulation to 
model the response of the detector, taking into account 
the varying accelerator and detector conditions, and to 
generate large samples of signal and background for the 
CP and control modes considered in the analysis. 



B. Event reconstruction and selection 

The B~ candidates are formed by combining a 
candidate with a track identified as a kaon [20] or with 
a K*~ candidate formed as a combination of a K'^ and 
a negatively charged pion, with an invariant mass within 
55 MeV/c^ of the nominal K*~ mass. Here and in the 
following, nominal mass values are taken from [21]. The 
l)" candidates are selected by requiring the K^n^TT~ or 
K^K^K~ invariant mass to be within 12 MeV/c^ of the 
nominal mass, and the momentum in the center-of- 
mass (CM) frame to be greater than 1.3 GeV/c. The 
tracks in K2.K^K~ are required to be pos- 

itively identified as kaons in the DCH and DIRC. The 
■K^ candidates from D^w^ decays are formed 

from pairs of photons with invariant mass in the range 
[115, 150] MeV/c^, and with photon energy greater than 
30 MeV. Photon candidates from D*^ — » D'^'y decays are 
selected if their energy is greater than 100 MeV. The Z)° 
candidates are combined with a 7r° (7) to form the 
candidate, and are required to have a D*^-D^ mass dif- 
ference within 2.5 (10) MeV/c^ of its nominal value. The 
Kg candidates are reconstructed from pairs of oppositely- 
charged pions constrained to originate from the same 
point and with an invariant mass within 9 MeV/c^ of 
the nominal mass. The cosine of the collinearity an- 
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glc between the K'^ momentum and the line connecting 
its parent particle (the or the K*~) and the decay 
points in the plane transverse to the beam, is required to 
be larger than 0.990 (0.997 for K° from K*' decays). 
This cut helps to significantly reduce background con- 
tributions from TTTTTTTr decays and from mis- 
reconstructed as K*~. The kinematic variables of the 
£)(*)o, K°, and 7r° when forming the B' , D°, and D*°, 
respectively, are fitted with masses constrained to nom- 
inal values. For B~ — > D^K*~ decays we also require 
I cos6h\ > 0.35, where 6h is the angle between the mo- 
mentum of the K*^ daughter pion and the parent B~ in 
the K*~ rest frame. The distribution of cosOh is propor- 
tional to cos^ 6h for B~ D^K*~ while it is approx- 
imately flat for e+e~ qq, q = u,d,s,c (continuum) 
background. 

We characterize B mesons using two almost inde- 
pendent variables, the beam-energy substituted mass, 
rriES = y/ + po • Pb)^/-E^ - Pb, and the energy 
difference AE = Eg - Eq/2, with p = {E,p), where the 
subscripts and B refer to the initial e~^e~ system and 
the B candidate, respectively, and the asterisk denotes 
the CM frame. The signal events peak at the B mass in 
mEs and at zero in AE. The m-ES resolution is about 
2.6 MeV/c^ and does not depend on the decay mode or 
on the nature of the prompt particle (K~ or K*~ can- 
didate). In contrast, the AE resolution depends on the 
momentum resolution of the meson and the prompt 
particle, and ranges between 15 MeV and 18 MeV, de- 
pending on the decay mode. We select events with 
rriES > 5.2GeV/c2 and -80 MeV < A£; < 120 MeV. 
We discriminate against the main background contribu- 
tion coming from continuum events through the fit to the 
data, as described in Sec. II C. 

For events in which multiple B candidates satisfy the 
selection criteria, the one whose measured D° mass dif- 
fers from the nominal value by the least number of stan- 
dard deviations, is accepted as signal candidate. For 
B~ —> D^K*~ decays we select the candidate with the 
smallest value for the sum of the squares of the diflfer- 
ences from nominal values, in standard deviations, of 
both K*^ and masses. The fraction of events in 
which we reconstruct more than one candidate is less 
than 1% for B' D^*^°K- samples and about 6% for 
B~ — » D^K*~. The cross- feed among the different sam- 
ples is negligible except for B^ D*'^[D'^j]K^ , where 
the background from B^ D*'^[D"Tr"]K^ is below 5% 
of the signal yield. If both B' D*°{D°tt°]K- and 
B~ —>■ D*^[D^'y]K~ candidates are selected in the same 
event, only the D*°[D"t:°]K- is kept. This con- 

tamination has a negligible effect on the measurement of 
the CP parameters. 

Figure 2 shows the jties distributions in the AE signal 
region defined through the requirement \AE\ < 30 MeV, 
after all selection criteria are applied. The reconstruc- 
tion efficiencies arc 20%, 9%, 12%, and 12%, for B' 
b°K', B- b*^[b^TT°]K~ , B~ D*°[i)°-f]K- , 



and B — > D'^K* decay modes, respectively, with 
b° K°Tr+n-. Similarly, for D° K°K+K- chan- 
nels we obtain 19%, 8%, and 11% (S" D^K*' is not 
reconstructed). 

The same selection criteria are applied to select — > 
£)(*)07^- control samples, apart from the particle iden- 
tification requirement on the prompt track, which is re- 
placed by a kaon identification veto. Since we evaluate 
AE with the kaon mass hypothesis, the AE distribu- 
tions are shifted by approximately -|-50 MeV, as given by 
Eq. (4). B~ — > £>°oj~ candidates are reconstructed simi- 
larly to B- D^K*-, with p(770)"7r^ candidates 
made using combinations of three charged tracks with the 
requirements that the oj~ invariant mass must be in the 
range [1.0,1.6] GeV/c^, and that of the p{770f within 
150 MeV/c^ of its nominal mass. As for signal samples, 
we select events with -80 MeV < AE" < 120 MeV. Fig- 
ure 3 shows the tues distributions for all control samples 
in the AE signal region defined through the requirement 
20 MeV < AE < 80 MeV, after aU selection criteria. 
The corresponding reconstruction efficiencies are similar 
to those estimated for the CP samples. 



C. Background composition and signal yields 

The largest background contribution is from contin- 
uum events, where a fake or true D^*^^ is combined 
with a random track {B~ D^*^^K~ samples), or a 
fake or true is combined with a random or fake 
K* {B~ — » D^K*~ sample). To separate continuum 
from BB events in a likelihood fit (discussed below and 
in Sec. IV), variables that characterize the event shape 
are used. We construct a Fisher discriminant [22] 
from a linear combination of four topological variables: 
the monomials Lq = J2iPi ^^'^ ^2 = Z^iPil cos0*|^, 
I cos I, and |cos^^|. Here, p* and 9* are the CM mo- 
mentum and the angle of the remaining tracks and clus- 
ters in the event, with respect to the B candidate thrust 
axis [23] . 0^ is the angle between the thrust axis of the B 
candidate and that of the rest of the event, and 6% is the 
polar angle of the B candidate momentum, in the CM 
frame. The first three variables account for the jet-like 
shape of continuum events, in comparison to the spher- 
ical topology of BB events. In particular, the variable 
I cos 6^ I peaks close to one for continuum while for BB it 
is essentially uniformly distributed. The angular distri- 
bution of the variable | cos 6^ \ follows 1 — cos^ 9^ for BB 
events and 1-t-cos^ 9g for e+e" — * qq [24]. The strategy of 
using the Fisher discriminant in a likelihood fit enhances 
significantly, typically about 25%, the signal reconstruc- 
tion efficiency compared to our previous analysis [18], 
where we required jcosfl^j < 0.8. At the same time it 
provides a larger sample of continuum events in the toes 
sidebands, thus allowing the determination of the back- 
ground properties directly from data (see Sec. IV). 

Another source of background is related to BB decays 
where a fake or true is combined with a random or 
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FIG. 2: (color online). Distributions of mES in A_E signal 
region for (a,e) B' D°K-, (b,f) -» 
(c,g) B- -> ZJ'OfL'^lTi'-, and (d) B' D^K*-, with (a- 
d) D° K°s-K+TT- and (e-g) D° KIK+K'. The curves 
superimposed represent the projections of the fit described in 
Sec. II C: signal plus background (solid black lines), the con- 
tinuum plus BB background contributions (dotted red lines), 
and the sum of the continuum, BB, and K/tt misidentifica- 
tion background components (dashed blue lines). 



FIG. 3: (color online). Distributions of jties in /S.E signal 
region for (a,e) B" -> £>°7r-, (b,f) B' -> £»*''[£)°7r°]7r-, 
(c,g) B- ^ b'^y&'y]-!^- , and (d) B" -* D°a-, with (a-d) 
D" K'^n+n- and (e-g) D° K'^K+K'. The curves su- 
perimposed represent the projections of the fit described in 
Sec. II C: signal plus background (solid black lines), the con- 
tinuum plus BB background contributions (dotted red lines), 
and the sum of the continuum, BB, and K/n misidentifica- 
tion background components (dashed blue lines). 



misidentified track or K*. The main single contribution 
for B- i)(*)OA'- signal comes from B- D'-*^°tt- 
decays when the pion is misidentified as a kaon. This 
source is accounted for separately from other BB back- 
grounds. This contribution can be discriminated from 
the signal due to the shift in the AE distribution rela- 
tive to that of signal events. Since AE is computed by 



assigning the kaon mass hypothesis to the prompt track, 
it is shifted by a quantity 

ASshift = 7PEP-II (^^ml + \p\^ - ^ml + |pP^ , (4) 

which depends on the momentum p of the prompt 
track in the laboratory frame and the Lorentz parameter 
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7PEP-II characterizing the boost of the CM relative to 
the laboratory frame, estimated from the PEP-II beam 
energies. For B~ — + D^K*~ signal the main BB back- 
ground source comes from B~ —y D°a^ decays. Since 
this contribution is highly suppressed by the cut on the 
cosine of the coUinearity angle at 0.997, it is not treated 
separately from other BB backgrounds. Non-K* decays 
contributing to the B~ D^K*~ sample are considered 
as signal, and their effect is accounted for by the factor 
K, defined in Eq. (3). 

We fit the seven signal samples B~ — > D^*^^K~ and 
B~ b^K*~ , and their control samples B~ — > D^*'>^it~ 
and B~ D^cii, using an unbinned extended maxi- 
mum likelihood method to extract signal and background 
yields, and probability density functions (PDFs) for the 
variables tues, AiJ, and the discriminant, in the AE 
selection region. Three different background components 
are considered: continuum events, K/tt misidentification 
(for B- D^*^^'K- and B' D^*^°7r- samples only), 
and other T(4S') BB decays. The log-likelihood for 
each of the CP and control samples is 



ln£ = 



El- 



(5) 



where Uj = {m^s, AE, J-}j characterizes the event j. 
Here, Pc(u) = Pc{mEs)'Pc{AE)Pc{^) is the combined 
selection PDF, verifying the normalization condition 
/ 7'c(u)du = 1, Nc the event yield for signal or back- 
ground component c, and rj = ^f.Nc. 

The signal toes distributions for each CP sample and 
its corresponding control sample are parameterized using 
a common single Gaussian. Similarly, the J- PDF makes 
use of a double Gaussian with different widths for the 
left and right parts of the curve (bifurcated Gaussian), 
and is assumed common for all CP and control samples. 
The signal AE distribution for B~ D^*^^^K^ events is 
parameterized with a double Gaussian function, while for 
B~ — > D^*^^Tr~ events we use the same function, shifted 
event-by-event using Eq. (4). For B~ D^K*~ and 
B~ D^a^ signal events a common double Gaussian is 
used instead. 

The continuum background in the jties distribution 
is described by a threshold function [25] while the con- 
tinuum AE distribution is described using a first order 
polynomial parameterization. The free parameters are 
different for each CP sample but common to the corre- 
sponding control sample. The J- distribution for contin- 
uum background is parameterized with the sum of two 
Gaussian functions and assumed common for all samples. 

The shape of the toes distribution for T{AS) BB 
background (excluding the K/ir misidentification contri- 
bution, as indicated previously) is taken from generic BB 
simulated events for each CP and control sample inde- 
pendently, and uses a threshold function [25] to describe 
the combinatorial component plus a bifurcated Gaus- 
sian to parameterize the contribution peaking at the B 
mass. The fraction of the peaking contribution is ex- 



tracted directly from the fit to the data, except for the 
K^K^K^ CP samples, where it is taken from 
the generic BB MC due to lack of statistics. The AE 
distribution for BB background is taken similarly from 
simulation and is parameterized with the sum of a second 
order polynomial and an exponential function that takes 
into account the increase of combinatorial background 
at negative AE values. A Gaussian function is also in- 
cluded to account for potential AE peaking background, 
although we find no significant peaking structure in any 
of our samples. The T distributions for signal and con- 
trol samples, and for the generic BB background, are 
assumed to be the same as found in the simulation. 

The selection fit yields, respectively, 610 ±34, 156 ±17, 
114 ± 16, and 110 ± 15 signal candidates, for B~ 
D°K^, B- D*^[D'^n"]K-, B" ^ D*°[D°-f]K-, and 
B- D°K*- reconstructed in the D° K°Tr+7r- 
mode, in agreement with expectations based on mea- 
sured branching fractions and efficiencies estimated from 
Monte Carlo simulation. Similarly, for K^K'^K~ 
channels we obtain, respectively, 132 ± 14, 35 ± 7, and 
16 ± 6. The corresponding signal yields for control sam- 
ples are 8262 ± 105, 2227 ± 55, 1446 ± 53, and 2321 ± 75, 
for £>o ^ K^^-K+TT- decay modes, and 1402±41, 350±20, 
and 236 ± 20, for K^K+R-. All errors are sta- 

tistical only. The curves in Fig. 2 and Fig. 3 represent 
the fit projections on the toes variable, for the AE signal 
region. 



III. r>° K^TT+n- AND D° 
AMPLITUDES 



A. Selection of flavor-tagged D" mesons 



The 



and £1° 



K^K+K^ (referred 
to hereafter collectively as K^h'^h^) decay am- 

plitudes are determined from Dalitz plot analyses of 
mesons from D^-k^ decays produced in e+e~ — > 

cc events. The charge of the low momentum tt"*" from 
the D*+ decay identifies ("tags") the flavor of the D'^. 
Reconstruction and selection of — > K^h'^h~ candi- 
dates from D*"*" D^TT"*" decays are similar to those 
from B^ D^*^^K~ decays, the only exception be- 
ing the kaon identification of only one charged kaon for 
D° K^K+K-. The D*+ candidates arc formed by 
combining the D''^ with the low momentum charged track. 
The two D*^ decaying daughters are constrained to orig- 
inate from the same point inside the PEP-II luminous re- 
gion. To reduce combinatorial background and contami- 
nation from BB decays, the i?" candidates are required 
to have a CM momentum greater than 2.2 GeV/c. 

Each D*^ sample is characterized by the distributions of 
two variables, the invariant mass of the candidate tod 
and the Am = D*^ — mass difference. We select 
candidates within ±0.64 MeV/c^ and ±0.61 MeV/c^, cor- 
responding to ±2 standard deviations, aroimd the nom- 
inal Am [21], for 



KliT+iT- and D° 
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respectively. Figure 4 shows the resulting D° Kgh^h~ 
mass distributions. The mjy lineshape is described using 
a two Gaussian function for the signal and a linear back- 
ground, as also shown in Fig. 4. The nio resolutions are 
6.7 MeV/c2 and 3.9 MeV/c^, for D° if^^+Tr" and 
D'^ K^K^K^ . The mass resolution for the latter 
is better than that of the former because of the much 
smaller Q-value involved. The signal purity in the signal 
box (±2(7 cutoff on mu, where cr stands for the mo reso- 
lution) is 97.7% and 99.3%, with about 487000 and 69000 
candidates, for XOtt+tt" and -> K'^K+R-. 

The Dalitz plot distributions for these events are shown 
in Fig. 5, with to^ = m^o^^ and ttiq — r7i^+^„ . 




m,, (GeV/c^) iTio (GeV/c^) 



FIG. 4; (color online). D mass distributions after all selec- 
tion criteria, for (a) D*+ D°tv+ , D° K'^tt+tt' and (b) 
D*+ -> D°TT+, D" ^ KIK+R- . The curves superimposed 
represent the result from the m_D fit (solid blue lines) and the 
linear background contribution (dotted red Unes). 
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m?(GeV2/c«) n^(GeV'/c*) 

FIG. 5: (color online). Dalitz plot distributions for (a) D'^ — > 
Kl-K+TT- and (b) D° ^ K^K+R- from D*+ events 
after all selection criteria, in the mass signal signal region. 
The contours (solid red lines) represent the kinematical limits 
of the D° Rl-K+'R- and ^ RIR+ R- decays. 



B. Dalitz plot analysis 

Three-body charm decays are expected to proceed 
through intermediate quasi-two body modes [26] and this 
is the observed pattern. We therefore use, as a baseline 
model to describe Ad ("^^ , m\ ) , an isobar approach con- 
sisting of a coherent sum of two-body amplitudes (sub- 



script r) and a "non-resonant" (subscript NR) contribu- 
tion [27], 

Ad (m) = are'*" A (m) -t- a^^e'*-^ , (6) 

r 

where we have introduced the notation m = (m?.,m^). 
The parameters (anr) and 4>r (0nr) are the magni- 
tude and phase of the amplitude for component r (NR). 
The function Ar = Fd 'x Fr 'x Tr 'x Wr is a Lorentz- 
invariant expression that describes the dynamic prop- 
erties of the meson decaying into K^h'^h~ through 
an intermediate resonance r, as a function of position in 
the Dalitz plane. Here, Fd (Fr) is the Blatt-Weisskopf 
centrifugal barrier factor for the D (resonance) decay 
vertex [28] with radius R = 1.5 GeV~^hc = 0.3 fm, 
T,. is the resonance propagator, and Wr describes the 
angular distribution in the decay. For Tr we use a 
relativistic Breit-Wigner (BW) parameterization with 
mass-dependent width [27], except for r = p(770)" and 
^(1450)*^ resonances where we use the Gounaris-Sakurai 
functional form [29]. The angular dependence Wr is 
described using either Zemach tensors [30, 31] where 
transversality is enforced or the helicity formalism [32- 
34] when we allow for a longitudinal component in the 
resonance propagator (see Ref. [27] for a comprehensive 
summary). Mass and width values are taken from [21], 
unless otherwise specified. 

The complex tttt S-wave dynamics in the D'^ — > 
ifgTT+Tr^ reaction [35], with the presence of several broad 
and overlapping scalar resonances, is more adequately 
described through the use of a K-matrix formalism [36] 
with the P-vector approximation [37]. This approach of- 
fers a direct way of imposing the unitarity constraint of 
the scattering matrix, not guaranteed in the case of the 
isobar model. The Dalitz plot amplitude Ad{^) given 
by Eq. (6) is then modified as 

AD{m) - Fi(5) + ar.e^^'-A.(m) + aNRe**"=^, (7) 

where Fi (s) is the contribution of tttt S-wave states writ- 
ten in terms of the K-matrix formalism, 

Fu{s) = ^ [/ - ^Kis)pis)]:^ P^s). (8) 

Here, s = uiq is the squared invariant mass of the tt+tt" 
system, / is the identity matrix, K is the matrix describ- 
ing the S-wave scattering process, p is the phase-space 
matrix, and P is the initial production vector (P-vector). 
The index u (and similarly v) represents the u"^ channel 
(1 = TTTT, 2 — KK, 3 — TTTTTTTr, 4 = rjrj, 5 — rjrj'). In this 
framework, the production process can be viewed as the 
initial preparation of several states, which are then prop- 
agated by the [/ — iK{s)p{s)]~^ term into the final one. 
The propagator can be described using scattering data. 
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provided that the two-body system in the final state is 
isolated and does not interact with the rest of the final 
state in the production process. The P-vector has to be 
determined from the data themselves since it depends on 
the production mechanism. Only the Fi amplitude ap- 
pears in Eq. (7) since we are describing the tttt channel. 
See Sec. Ill C for more details. 

The decay amplitude .4_D(m) is then determined from 
a maximum likelihood fit to the I?" K^h^hr Dalitz 
plot distribution m in a ±2a cutoff region of the mass, 
with log-likelihood function 



We adopt the same parameterizations for K , p, and P 
in Eq. (8) as in Refs. [27, 39, 40]. For the K matrix we 
have 



9u 9v 



scatt 
uv 



1 



/ao(s),(11) 



where is the coupling constant of the K-matrix pole 

rUa to the u^^ channel. The parameters /u"^,^** and .Sq'^'^** 
describe the slowly-varying part of the K-matrix. The 
factor 



E 



In [/sigI?sig,T(mi) + (1 - /sig)2?bkg,T(mj)] ,(9) 



where /gig represents the fraction of signal obtained from 
the fit to the mass spectrum, and 2?sig(bkg),:F(mj) is the 
signal (background) Dalitz plot PDF for event j, sat- 
isfying the condition J I?sig(]3ijg-)^(m)dm = 1. For 
signal events, Psig,+ (m) = |^£)(m)pe(m), while for 
r>sig,-(m) = |^D(m)pe(m), with m = (m+,m?.). Here 
e(m) represents the efficiency variations on the Dalitz 
plot, evaluated using high statistics signal MC samples. 
These are generated according to a uniform distribution 
and parameterized using third-order polynomial func- 
tions in two dimensions, symmetric for D° — > if^TT+TT" 
and asymmetric for K'^K~^K~ to account for pos- 

sible charge asymmetries in the K~ and detection 
efficiencies. The Dalitz plot distributions for the back- 
ground, Dbkg,q=(m), are determined using mass side- 
band data. 

For each contribution r we evaluate the fit fraction 
as the normalized integral of a^|^r(m)P over the Dalitz 
plane [27], 



fr 



aj J |A(m)|2dm 



(10) 



The sum of fit fractions does not necessarily add up to 
unity because of interference effects among the ampli- 
tudes. 



Dalitz model 



The P- and D-waves of the — > i^gTr+Tr" decay am- 
plitude are described using a total of 6 resonances leading 
to 8 two-body decay amplitudes: the Cabibbo allowed 
(CA) K*{892)-, K*{1680)-, i^2*(1430)-, the doubly- 
Cabibbo suppressed (DCS) K*{892)+, ii:*(1430)+, and 
the CP eigenstates /o(770)°, w(782), and /2(1270). Since 
the Ktt P-wave is largely dominated by the _ft'*(892)^, 
the mass and width of this resonance are simultane- 
ously determined from our fit to the tagged sam- 



ple, 



A:*(892)T 



893.61 ±0.08 MeV/c^ and Tk*{s92)i= 



46.34 ± 0.16 MeV/c^ (errors are statistical only). The 
mass and width values of the K*{1680)~ are taken 
from [38] , where the interference between the Kn S- and 
P-waves is properly accounted for. 



1-SAo f ml 
s- Sao \ 2 



(12) 



suppresses the false kinematical singularity at s = in 
the physical region near the nn threshold (the Adler 
zero [41]). The parameter values used in this analysis are 
listed in Table I, and are obtained from a global analy- 
sis of the available tttt scattering data from threshold up 
to 1900 MeV/c2 [39]. The parameters f^l^^\ for u 1, 
are all set to zero since they are not related to the tttt 
scattering process. Similarly, for the P vector we have 



pprod 



l-s; 



prod 




S — S, 



prod * 



(13) 



Note that the P-vector has the same poles as the K- 
matrix, otherwise the Fi vector would vanish (diverge) 
at the K-matrix (P-vector) poles. The parameters 
/f ^'^ and Sq''°'^ of the initial P-vector are obtained from 
our fit to the tagged D° i^°7r+7r~ data sample. 



TABLE I: K-matrix parameters from a global analysis 
of the available tttt scattering data from threshold up to 
1900 MeV/c^ [39]. Masses and coupling constants are given 
in GeV/c^ 











9vv 


9nn' 


0.65100 


0.22889 


-0.55377 


0.00000 


-0.39899 


-0.34639 


1.20360 


0.94128 


0.55095 


0.00000 


0.39065 


0.31503 


1.55817 


0.36856 


0.23888 


0.55639 


0.18340 


0.18681 


1.21000 


0.33650 


0.40907 


0.85679 


0.19906 


-0.00984 


1.82206 


0.18171 


-0.17558 


-0.79658 


-0.00355 


0.22358 


„scatt 
So 


/■scatt 
Jll 


/?scatt 
/l2 


/?scatt 
/13 


/■scatt 
/l4 


/!scatt 
/15 


-3.92637 


0.23399 


0.15044 


-0.20545 


0.32825 


0.35412 


SAO 


SA 










-0.15 


1 











For the Kir S-wave contribution to Eq. (7) we use a pa- 
rameterization extracted from scattering data [38] which 
consists of a ii:^(1430)- or i<:^(1430)+ BW (for CA or 
DCS contribution, respectively) together with an effec- 
tive range non-resonant component with a phase shift. 



Ak^ L=o{ra) = FsindFe^^" + Rsm6Re''"'e''^''^ , (14) 
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with 



Sr = 
6f = 



tan 



bp + cot 



M2 
1 

aq 



rq 

y 



(15) 



The parameters a and r play the role of a scattering 
length and effective interaction length, respectively, F 
{4>f) and R {(pii) arc the amplitudes (phases) for the non- 
resonant and resonant terms, and q is the momentum 
of the spectator particle in the Ktt system rest frame. 
Note that the phases Sp and depend on m|-^. M 
and r(m|-^) are the mass and running width of the res- 
onant term. This parameterization corresponds to a K- 
matrix approach describing a rapid phase shift coming 
from the resonant term and a slow rising phase shift gov- 
erned by the non-resonant term, with relative strengths 
R and F [42]. The parameters M, T, F, (j>F, R, (I>r, 
a and r are determined from our fit to the tagged 
sample, along with the other parameters of the model. 
Other recent experimental efforts to improve the descrip- 
tion of the K-K S-wave using K-matrix and model inde- 
pendent parameterizations from high-statistics samples 
of Z)+ ii'~7r+7r+ decays are described in Ref. [43]. 

Table II summarizes the values obtained for all free pa- 
rameters of the £>° ii:°7r+7r- Dalitz model: CA, DCS, 
and CP eigenstates complex amplitudes arC^'f''', tt+tt" S- 
wave P-vector parameters, and Ktt S-wavc parameters, 
along with the fit fractions. The non-resonant term of 
Eq. (7) has not been included since the rnr and Kn S- 
wave parameterizations naturally account for their re- 
spective non-resonant contributions. The fifth P-vector 
channel and pole have also been excluded since the rjij' 
threshold and the pole mass TO5 arc both far beyond 
our TTTT kinematic range, and thus there is little sensi- 
tivity to the associated parameters, f^^"^^ and /S^, re- 
spectively. The amplitudes are measured with respect to 
Kgp{770)'^ which gives the second largest contri- 
bution. Wc report statistical errors only for the ampli- 
tudes, but for the fit fractions we also include system- 
atic uncertainties (see Sec. HIE), which largely dom- 
inate. The Ktt and tttt P-wavcs dominate the decay, 
but significant contributions from the corresponding S- 
waves are also observed (above 6 and 4 standard devi- 
ations, respectively). We obtain a sum of fit fractions 
of (103.6 ± 5.2)%, and the goodness of fit is estimated 
through a two-dimensional test performed binning the 
Dalitz plot into square regions of size 0.015 GeV^/c^, 
yielding a reduced of l-H (including statistical er- 
rors only) for 19274 degrees of freedom. The variation of 
the contribution to the as a function of the Dalitz plot 
position is approximately uniform. Figure 6(a,b,c) shows 
the Dalitz fit projections overlaid with the data distribu- 
tions. The Dalitz plot distributions are well reproduced, 
with some small discrepancies in low and high mass re- 
gions of the mg projection, and in the ^(770)*^ — a;(782) 
interference region. 



TABLE II: CA, DCS, and CP eigenstates complex ampli- 
tudes fflr-e"*'', TTTT S-wave P-vector parameters, Kn S-wave 
parameters, and fit fractions, as obtained from the fit of 
the D° K'in+TT~ Dalitz plot distribution from D*+ 

D°n~^. P-vector parameters /i^™'', for v ^ 1, axe defined as 
fiv°'^/fii°'^- Errors for amplitudes are statistical only, while 
for fit fractions include statistical and systematic uncertain- 
ties, largely dominated by the latter. Upper limits on fit frac- 
tions are quoted at 95% confidence level. 



Component 




<^r (dcg) 


Fraction (%) 


i^*(892)- 


1.740 ± 0.010 


139.0 ±0.3 


55.7 ±2.8 


i^o(1430)- 


8.2 ±0.7 


153 ±8 


10.2 ± 1.5 


K2(1430) 


1.410 ± 0.022 


138.4 ± 1.0 


2.2 ± 1.6 


K (iboUj 


"I AG _l_ r\ 1 n 
i.4D ± O.iU 


1 'TA _1_ A 


n '7 _l_ 1 n 
U.I ± 1.9 




0.158 ±0.003 


- 42.7 ± 1.2 


U.^O U.ZO 




0.32 ± 0.06 


143 ± 11 


< 0.05 


Al(ii:!0) 


().()9i ± ().()!() 


8.") ± li 


< 0.12 


P\i '"J 


1 





91 n -1- 1 fi 




0.0527 ±0.0007 


126.5 ±0.9 


u.y It i.u 


£ /■io'7r>\ 


0.606 ± 0.026 


157.4 ± 2.2 


U.D ± U. / 


Pi 


9.3 ±0.4 


-78.7 ±1.6 




P2 


10.89 ± 0.26 


-159.1 ±2.6 




fj 
P3 


24.2 ± 2.0 


168 ±4 




/?4 


9.16 ±0.24 


90.5 ± 2.6 




(.prod 
111 


7.94 ±0.26 


73.9 ±1.1 




r prod 
./l2 


2.0 ±0.3 


- 18 ±9 




r prod 

Irs 


5.1 ±0.3 


33 ±3 




r prod 
Jl4 


3.23 ±0.18 


4.8 ±2.5 




prod 

■'o 


-0.07 ±0.03 




TTTT S-wave 






11.9 ±2.6 


M (GcV/c^) 
P ( GeV/c^) 


1.463 ± 0.002 




0.233 ±0.005 




F 


0.80 ± 0.09 




4>F 


2.33 ±0.13 




R 




1 




4>R 


-5.31 ±0.04 




a 


1.07 ±0.11 




r 


-1.8 


±0.3 





As a cross-check, we alternatively parameterize the tttt 
and Ktt S- waves using the isobar approximation with the 
following BW amplitudes (plus the non-resonant contri- 
bution): the CA K^{U30)-, the DCS K^{U30)+, and 
the CP eigenstates /o(980), /o(1370), cr and an ad hoc 
a'. This model is very similar to that used in our pre- 
vious measurement of 7 [18], except that the /r*(1410)~ 
and /!)(1450)° resonances have been removed because of 
their negligible fit fractions. Masses and widths of the a 
and cr' scalars arc obtained from the fit, = 528 ± 5, 
F,, = 512 ± 9, M^/ = 1033 ± 4, and T^, = 99 ± 6, given 
in MeV/c^. Mass and width values for the K^{U30)'f, 
/o(980), and /o(1370) arc taken from [44, 45]. We ob- 
tain a sum of fit fractions of 122.5%, and a reduced 
of 1.20 (with statistical errors only) for 19274 degrees of 
freedom, which strongly disfavors the isobar approach in 
comparison to the K-matrix formalism. 



12 




1.6 1.8 

m? (GeV^/c") 



1.6 1.8 

ml (GeV^/c") 



mg (GeVV) 



FIG. 6: (color online). D" ^ K^K^h Dalitz plot projections from D*'^ — > D^tt"*" events on (a,d) m?., (b,e) m\, and (c,f) ttiq, 
for (a,b,c) D° K'^^tv+tt- and (d,e,f) D° K^K+K'. The curves are the reference model fit projections. 



D. 



KlK+K- 



Dalitz model 



The description of the K^K^K decay ampli- 

tude uses Eq. (6) and consists of five distinct resonances 
leading to 8 two-body decays: K^ao(980)°, ii:^0(lO2O), 
iC-ao(980)+, Klfo{mQ), K+ao{9?>Q)-, ^0/2(1270)", 
i(:°ao(1450)^ and ii'-ao(1450)+. This isobar model is 
essentially identical to that used in our previous analy- 
sis of the same reaction [46], but for the addition of the 
ao(1450) scalar, whose contribution is strongly supported 
by the much larger data sample, as well as of a D-wave 
contribution parameterized with the /2(1270) tensor. At- 
tempts to improve the model quality by adding other 
contributions (including the non-resonant term) did not 
give better results. 

The 0(1020) resonance is described using a relativistic 
BW, with mass and width left free in our fit to the D'^ 
tagged sample in order to account for mass resolution ef- 
fects. The ao(980) resonance has a mass very close to the 
KK threshold and decays mostly to rj-K. Therefore it is 
described using a coupled channel BW [27, 46], where 
the mass pole and coupling constant to rjir are taken 
from [47], while the coupling constant to KK, 9kk^ 
determined from our fit. 

Table III summarizes the values obtained for all free 
parameters of the — > K^K~^K~ Dalitz model, the 
complex amplitudes are*"^'', the mass and width of the 



0(1020) and the coupling constant 5^^, together with 
the fit fractions. The value of is consistent with 
our previous result [46] , and differs significantly from the 
measurement reported in [47]. All amplitudes are mea- 
sured with respect to D° ^^300(980)°, which gives the 
largest contribution. The sum of fit fractions is 152.3%, 
and the reduced is 1-09 (with statistical errors only) 
for 6856 degrees of freedom, estimated from a binning of 
the Dalitz plot into square regions of size 0.045 GeV^/c^. 
The variation of the contribution to the s-s a function 
of the Dalitz plot position is approximately uniform in 
the regions where most of the decay dynamics occurs. 
Figure 6(d,e,f) shows the fit projections overlaid with the 
data distributions. The Dalitz plot distributions are well 
reproduced, with some small discrepancies at the peaks 
of the and projections. 



E. Systematic uncertainties 

Systematic uncertainties on Aoijn) are evaluated by 
repeating the fit to the tagged D'-^ samples with alter- 
native assumptions to those adopted in the reference 
£)" ^ K°n+T:- and Z?" ^ K°K+K- amplitude analy- 
ses. These uncertainties can then be directly propagated 
to the measurement of the CP parameters, as discussed in 
Sec. IV B, and the total systematic error can be obtained 
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TABLE III: CP eigenstates, CA, and DCS complex ampli- 
tudes fflT-e"*'' and fit fractions, obtained from the fit of the 
Z)° KlK+K- Dalitz plot distribution from D*+ ->■ D°ir+. 
The mass and width of the 0(1020), and the g^j^ coupling 
constant are simultaneously determined in the fit, yielding 
M^(i020) = 1.01943 ± 0.00002 GeV/c^ r^(io2o) = 4.59319 ± 
0.00004 MeV/c^ and = 0.550±0.010 GeV/c^ Errors for 
amplitudes are statistical only. Uncertainties (largely domi- 
nated by systematic contributions) are not estimated for the 
fit fractions. 



Component 




4>r (deg) 


Fraction (%) 


7^i'a(,(980)" 


1 





55.8 


ii:g(/)(1020) 


0.227 ± 0.005 


-56.2 ±1.0 


44.9 


^s/o(1370) 


0.04 ± 0.06 


- 2 ±80 


0.1 


if (1270) 


0.261 ± 0.020 


-9±6 


0.3 


K°ao{U50)° 


0.65 ± 0.09 


- 95 ± 10 


12.6 




0.562 ±0.015 


179 ±3 


16.0 


K-ao{U50)+ 


0.84 ± 0.04 


97 ±4 


21.8 


K+ao{98Q)- 


0.118 ±0.015 


138 ±7 


0.7 



from the sum square of the individual contributions. In 

this paper we have also propagated these systematic un- 
certainties to the measurement of the D° — » i^T^Tr+Tr" fit 
fractions, as reported in Table II. In general, each of the 
considered alternative models has a reduced poorer 
than that of the reference. Therefore, our systematic un- 
certainties do not include potential contributions due to 
the residual poor quality of the reference model fit, as 
reported in Sees. IlIC and HID. 



1. Model contributions 

Dalitz model systematic uncertainties on Ad{i^) are 
related to the model dependence of the strong charm de- 
cay phase as a function of the Dalitz plot position when 
it is determined from the Dalitz plot density, which only 
depends on decay rates. 

We use alternative models where the BW parameters 
are varied according to their uncertainties or changed by 
values measured by other experiments. This is the case of 
the /o(1370), where the reference values [44] are replaced 
by alternative measurements [48], and the Jsr*(1680)~, 
where the reference parameters [38] are replaced by those 
from [21]. We also build models using alternative param- 
eterizations, as in the case of the /9(770)° where the refer- 
ence Gounaris-Sakurai form is replaced by the standard 
relativistic BW. 

To estimate the rnr S-wave systematic error we replace 
the reference K-matrix solution (Table 1) by all alter- 
native solutions analyzed in Ref. [39]. Analogously, the 
uncertainty on the parameterization of the Kw S-wave 
is estimated using a standard relativistic BW describing 
the Kq{14:30)^ with parameters taken either from [44] or 
simultaneously determined from our fit to the sample. 
Additionally, the isobar model is used as a cross-check of 
the combined tttt and Kn S-wave effect. 



Uncertainties due to our choice of the angular depen- 
dence arc estimated by replacing the reference Zemach 
tensors by the helicity formalism. The effect is neg- 
ligible for S- waves, very small for P- waves, but larger 
for D- waves [31]. Other alternative models are built 
by changing the Blatt-Weisskopf radius between and 
3 GeV~^hc, and removing and adding resonances with 
small or negligible fit fractions. For D° iiTgTr+Tr", 
we added p(1450)° and CA K*{U10)-. Similarly, for 
— > KgK^K~, we removed all the ao(1450) charged 
states and the /2(1270), and added the /o(980) and the 
charged DCS ao(1450). The /o(980) resonance is de- 
scribed using a coupled channel BW with parameters 
taken from a variety of experiments [45, 48, 49]. 



2. Experimental contributions 

Experimental systematic errors come from uncertain- 
ties in the knowledge of variations of the reconstruc- 
tion efficiency on the Dalitz plot, background Dalitz plot 
shapes, mass resolution, mistag rate, and binning. 

The uncertainty from the efficiency variations on the 
Dalitz plot e(m) has been evaluated assuming the effi- 
ciency to be flat. Tracking efHciency studies in data and 
MC show that this method gives a conservative estimate 
of the imperfections of the detector simulation, which ap- 
pear mainly at the boundaries of the phase space because 
of the presence of very low momentum tracks. 

Systematic errors related to the backgromid Dalitz plot 
profile 2?bkg,q:(ni) are determined assuming a flat shape, 
which gives the largest effect among other alternative 
profiles obtained using either the m.o sideband from con- 
tinuum MC, or the mo signal region from continuum MC 
after removal of true mesons. 

All the resonances, except for the a;(782) and (/)(1020), 
have intrinsic width significantly larger than possible bias 
on invariant mass measurement and resolution. We esti- 
mate the systematic uncertainty associated with lu{782) 
by repeating the model fit using an overall width result- 
ing from adding in quadrature its natural width and the 
mass resolution in the 782 MeV/c^ tttt mass region. No 
systematic error is assigned for the 0(1020), since the ref- 
erence model has been extracted with its mass and width 
as free parameters. 

The uncertainty due to a wrong identification of the 
flavor of the 1)° (D^) meson from the D*+ D°Tr+ 
{D*~ D°7r~) decay, due to the association of the D 
meson with a random soft pion of incorrect charge has 
been evaluated taking into account explicitly the rate of 
mistags observed in the MC, at 0.7% level. 

Effects from limited numerical precision in the com- 
putation of normalization integrals and binning in the 
Dalitz plane have been evaluated using coarser and thin- 
ner bins. 
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IV. DALITZ PLOT ANALYSIS OF 

D'^'^K- and B- DK'- DECAYS 



Once the decay amplitudes ^_D(m) for D° - ^'■s' 
and — > K'^K^K~ are known, they are fed into 
r^'*(m) and rsz[i(m). The extraction of the CP- violating 



parameters x^-*, 



(*) 



and UszT is then performed 



through a simultaneous unbinned maximum likelihood 
fit (referred to hereafter as the CP fit) to the r^''(m) 
and rsi|i(m) Dalitz plot distributions of the seven sig- 
nal modes, in the AE signal region defined as |A£'| < 
30 MeV. Figures 7 and 8 show these distributions sep- 
arately for and decays in a region enriched in 
signal through the requirements rngs > 5.272 GeV/c^ 
and J- > —0.1. The efficiency of the Fisher cut in the 
\AE\ < 30 MeV and wes > 5.272 GeV/c^ region is 
around 70% for signal events, while for continuum back- 
ground events it is below 1%. 

The log-likelihood function for each of the seven CP 
samples generalizes Eq. (5) to include the Dalitz plot 
distributions, 



ln£ = —T] + In 



(16) 



Here, Vc.^{m.j) is the Dalitz plot PDF for event j satisfy- 
ing the normalization condition J I^c.^f (m)dm = 1, and 
Ac accounts for any asymmetry in the absolute number 
of B~ and B^ candidates (charge asymmetry) for com- 
ponent c. 

For B=F D°K^ signal, Vsig,^{m) = r^(m)e(m), 
where the efficiency map in the Dalitz plot e(m) is de- 
termined as for Z3*+ £'°7r+ events (Sec. IIIB). We 



replace in Eq. (2) by 



ST 



yz-. The physical 



condition r^- = is recovered in the statistical pro- 
cedure to extract 7 from Xz^^y^, as discussed in Sec. V. 
The same procedure is applied analogously to the other 
signal samples. 

We consider the same background components as in 
the selection fit, with some important modifications. 
First, events falling into the continuum and BB back- 
ground components are divided into events with a real 
or a fake (combinatorial) D'^ meson. Dalitz plot shapes 
for fake D'^ mesons from continuum are extracted as 
described in Sec. IIIB, using events in the continuum 
enriched region (rngs and mo sideband regions), while 
those from BB are determined from MC events. Events 
containing a real are further divided into "right-sign" 
and "wrong-sign" flavor categories depending on whether 
they are combined with a negative or positive kaon (or 
K*). We pay special attention to this charge- flavor cor- 
relation in the background since it can mimic either the 
6 — > c or the 6 — s- u signal component. Second, we have 
included a background contribution due to signal events 
where the kaon (or K*) comes from the other B decay; 
this amounts to 9% of the B~ D^K*~ signal, but is 
negligible for B" D^*^"K-. 




m? (GeV^/c") 



2 3 
mi (GeV^/c*) 



FIG. 7: (color online). D° K^tv+tv' Dalitz plot dis- 
tributions for (a) B- -» D°K', (b) B+ D°K+, (c) 
B- D*°[D°TT°]K-, (d) B+ D*°[D°7v°]K+, (e) B' 
D*°[D°'y]K-, (f) B+ £>*°[f>°7]A'+, (g) B" ^ D^K*-, 
and (h) B^ — > LP K*^ , for the AJ5 signal region. The require- 
ments TTiES > 5.272 GeV/c^ and T > —0.1 have been applied 
to reduce the background contamination, mainly from con- 
tinuum events. The contours (solid red lines) represent the 
kinematical limits of the K^-k^-k" decay. 



The TTiES, and T PDF parameters in the CP 

fit are the same as those used in or obtained from the 
selection fit, except for the mss peaking fractions for 



K'tt'^tt channels, which are allowed to vary 



since their values depend on the AE region used for 
the fit. Other parameters simultaneously determined 
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FIG. 8: (color online). D° KIK+ K~ Dalitz plot dis- 
tributions for (a) B~ ^ D°K-, (b) B+ D'^K+, (c) 
B- D*°[D°TT°]K-, (d) B+ D'-°[D°7v°]K+, (e) B" ^ 
£)*°[L)°7]iC-, and (f) B+ £»'°[£i"7]7f+, for the A£ signal 
region. The requirements tties > 5.272 GeV/c^ and > —0.1 
have been applied to reduce the background contamination, 
mainly from continuum events. The contours (solid red lines) 
represent the kinematical limits of the K'^K'^ K~ de- 

cay. 



from the fit, along with the CP-violating parameters 



^ y'^\ Xs^, and Hs^, are: signal and background 
yields, signal charge asymmetries, and fractions of true 
mesons for all decay modes and right-sign fractions 
for — > K'^tt^tt^ channels in continuum background. 
Right-sign fractions for the modes with — > K'^K^K^ 
are fixed from MC simulation due to lack of statistics 
and the limited discriminating power between D'^ and 
Dalitz plot distributions. Similarly, fractions of true 
D° mesons and charge-flavor correlation for the BB com- 
ponent are determined using MC events, because of the 
lack of BB background statistics. 



A. Results and cross-checks 

We find 600±31, 133±15, 129±16, and 118±18 signal 
events, for B- D^R-, B- D*"[D°tt^]K- , B- 
D*°[D°'y]K-, and B- -> D"K*- decay modes, respec- 
tively, with D° K^TT+n-. Similarly, for the D° 
K°K+K~ channels we obtain 112±13, 32±7, and 21±7 
signal events, for B" D^R-, B- D*"[D"tt^]K- , 
and B- D*°[D°-f]K-. Errors are statistical only. 
No statistically significant charge asymmetries are ob- 
served. The results for the CP- violating parameters , 

, Xsip , and j/^ip , are summarized in Table IV. The 
only non-zero statistical correlations involving the CP 
parameters are for the pairs (xl,y!l), 
{xs-,ys-), {xs+,ys+), which amount to 0.4%, 
3.5%, -14.0%, -5.6%, -29.9%, and 6.8%, respectively. 
Figure 9 shows the 39.3% and 86.5% 2-dimensional 
confidence-level (CL) contours in the (xzp,?/zp), {x^,y^), 
and (xsip,?/sip) planes, corresponding to one- and two- 
standard deviation regions (statistical only). The sepa- 
ration of the B~ and B'^ positions in the {x, y) plane is 
equal to 2rB \ sm^\ and is a measurement of direct CP vi- 
olation. The angle between the lines connecting the B~ 
and i?+ centers with the origin (0, 0) is equal to 27. 

A variety of studies using data, parameterized fast 
Monte Carlo, and full GEANT4-simulated samples have 
been performed to test the consistency of the results and 
to verify the analysis chain and fitting procedure, as de- 
scribed below. 

The CP fit to the -^^ D^*'^°K- samples has been 
performed separately for D'~' if^Tr+Tr" and _D" 
K^K^K~ samples. Figure 10 shows the resulting one- 
and two-standard deviation regions in the (x^ ) 
planes. We find statistically consistent results between 
the different subsets. The same fitting procedure has 
been applied to the B^ Z?(*'°7r^ control samples. In 
this case we expect r^*l «| V^^V^^ \ / \ V^^V;^ \ cp 
to be approximately 0.01. Since the experimental resolu- 
tions on {x-^'.„,y^'^) are expected to have the same order 

of magnitude, the (xfj*''^, ylj*''^) contours for B^ and 5+ 
decays should be close to the origin up to ^ 0.01. De- 
viations from this pattern could be an indication that 
the Dalitz plot distributions are not well described by 
the models. Figure 11 shows the resulting one- and 
two-standard deviation regions for (x^''^, j/^*|), consis- 
tent with the expected values. Moreover, we find sta- 
tistically consistent results between the — > T^sTt+tt" 
and K'^K+K- samples. 

An additional test of the fitting procedure is performed 
with parameterized MC simulations consisting of about 
500 experiments generated with a sample size and com- 
position corresponding to that of the data. The CP pa- 
rameters are generated with values close to those found 
in the data and the reference CP fit is performed on each 
of these experiments. The r.m.s. of the residual distri- 
butions for all the CP parameters (where the residual is 
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TABLE IV: CP- violating parameters x-^ , , ajsip, and ys^, as obtained from the CP fit. The first error is statistical, the 
second is experimental systematic uncertainty and the third is the systematic uncertainty associated with the Dalitz models. 



Parameters 


B- D°K- 


B- D*°K- 


B- D°K'- 


X- 


x*_ 


, Xs — 


0.090 ± 0.043 ± 0.015 ± 0.011 


-0.111 ± 0.069 ± 0.014 ± 0.004 


0.115 ± 0.138 ± 0.039 ± 0.014 


y- 


y- 


Vs- 


0.053 ± 0.056 ± 0.007 ± 0.015 


-0.051 ± 0.080 ± 0.009 ± 0.010 


0.226 ± 0.142 ± 0.058 ± 0.011 


x+ 


xX 


, Xs + 


-0.067 ± 0.043 ± 0.014 ± 0.011 


0.137 ± 0.068 ± 0.014 ± 0.005 


-0.113 ± 0.107 ± 0.028 ± 0.018 


y+ 


y+ 


ys+ 


-0.015 ± 0.055 ± 0.006 ± 0.008 


0.080 ± 0.102 ± 0.010 ± 0.012 


0.125 ± 0.139 ± 0.051 ± 0.010 
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FIG. 9: (color online). Contours at 39.3% (dark) and 86.5% (light) 2-dimensional confidence-level (CL) in the (a) (a;z|:,y^), (b) 
{x^,y^), and (c) {xs^,ys^:) planes, corresponding to one- and two-standard deviation regions (statistical only), for B~ (thick 
and solid lines) and B^ (thin and dotted lines) decays. 



defined as the difference between the fitted and generated 
values) is found to be consistent with the mean (Gaus- 
sian) statistical errors reported by the fits. The mean val- 
ues of the residual distributions are consistent with zero. 
Only for Xg^ and yszp we observe small biases (at 10% 
level of the statistical uncertainty), as a consequence of 
the non-Gaussian behavior of samples with small statis- 
tics. This small deviation from Gaussian behavior is also 
observed in the data, as shown in Fig. 9(c). The sta- 
tistical errors on the CP parameters and the calculated 
correlation coefficients among them extracted from the 
fit are consistent with the range of values obtained from 
these experiments. We also observe that the fit errors are 
independent of the truth values. 



Finally, samples of signal and background GEANT4- 
simulated MC events with a full detector simulation are 
used to validate the measurement. We performed fits to 
signal samples, using the true and reconstructed B meson 
charge and D'^ Dalitz plot distributions, obtaining in all 
cases results consistent with those generated. 



B. Systematic uncertainties 

1. Dalitz model contributions 

Dalitz model uncertainties are evaluated by repeating 
the fit to the tagged samples with alternative assump- 
tions to those adopted in the reference D'^ K^tt^tt^ 
and D° K^K+R- amplitude analyses (Sec. HIE), 
and then are propagated to the CP parameters. To 
propagate each systematic uncertainty on Ad (m) to the 
CP parameters we have generated samples of 
ij(*)OK~ and B~ — > D^K*~ signal events that are one 
hundred times larger than each measured signal yield in 
data. These virtually infinite samples reduce to a negligi- 
ble level statistical differences between the models. The 
Dalitz plot distributions are generated according to 
the reference models and to CP parameters consistent 
with the values found in data. The CP parameters are 
then extracted by fitting the generated Dalitz plot dis- 
tributions using the reference or one of the alternative 
models. The difference is taken as the systematic uncer- 
tainty associated with each alternative model, and the 
sign of the variation is used to estimate whether the 
different contributions are positively or negatively cor- 
related (Appendix A). When two alternative models are 
built from an up and down variation of the same param- 
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FIG. 10: (color online). Contours at 39.3% (dark) and 
86.5% (light) 2-dimensional confidence-level (CL) in the (a,c) 
(a;=|=, j/zp) and (b,d) {x^,y^) planes, corresponding to one- and 
two-standard deviation regions (statistical only), for B~ 
73)(*)o^- (^jjij,]^ goii^j ijjjgg) 5+ ^ d'-*'>°K+ (thin 

and dotted lines) decays, for (a,b) -D" A"s7r^7r~ and (c,d) 
D° KgK^K~ only decay modes. 
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FIG. 11: (color online). Contours at 39.3% (dark) and 
86.5% (light) 2-dimensional confidence-level (CL) in the (a) 
(a;=|=,,r, j/:F,7r) and (b) (x^^^, y^^^) planes, corresponding to 
one- and two-standard deviation regions (statistical only) , for 
B- D(*)°7r" (thick and solid lines) and B+ D''*^°n+ 
(thin and dotted lines) control sample decays. Note the dif- 
ferences in scale when comparing to Figs. 9 and 10. 



eter, we take the maximum variation as the systematic 
error. Assuming the contributions are uncorrelated, we 
sum in quadrature to obtain the total systematic uncer- 
tainty. 



The statistical errors in the Dalitz model parameters 
obtained from the tagged -D° samples have been propa- 
gated to the CP parameters by repeating the CP fit with 
those parameters randomized according to their covari- 
ance matrix. 

Table V summarizes the main contributions from 
all the alternative models considered and discussed in 
Sec. HIE. Contributions from other models are found to 
be negligible. 

We have also evaluated the effect on the measured CP 
parameters when we parameterize the tttt and Kit S- 
waves in -fCgTr+Tr" using the isobar model instead 

of the K-matrix model (plus the non-resonant contribu- 
tion), as described in Sec. Ill C. The variations are found 
to be smaller than the sum of the tttt and Kt: S-wave sys- 
tematic uncertainties, and are used as a cross-check of the 
procedure adopted for assigning this contribution to the 
total Dalitz model error. 



2. Experimental contributions 

Experimental systematic uncertainties arise from sev- 
eral sources and their main contributions are summarized 
in Table VI. They are small compared to the statistical 
precision, and their sum is similar to the Dalitz model 
uncertainty. Other sources of experimental systematic 
uncertainty, e.g. the assumption of perfect mass reso- 
lution for the Dalitz plot variables m, are found to be 
negligible. 

Statistical uncertainties due to the tties, A£', and J- 
PDF parameters for signal and background extracted 
from the selection fit (fixed in the reference CP fit) are 
estimated by repeating the CP fit with PDF parameters 
randomized according to their covariance matrix. Possi- 
ble bias due to differences in the mES and J- shapes for 
continuum and BB background events between the AE 
selection and signal regions are evaluated applying the 
selection fit in the AE signal region. Other PDF param- 
eters, such as the mss end-point, BB AE peaking frac- 
tions. Toes BB peaking fractions for D'^ — > K'^K'^K^ 
channels, and PEP-II boost are varied by one standard 
deviation. We account for toes and AE differences in 
BB background for true and fake D mesons, while for 
continuum events we do not observe differences. We also 
find the effect of the small correlation between toes, AE, 
and J- variables negligible. 

The uncertainties related to the knowledge of the 
fractions for the small BB background are estimated 
from the maximum variations of the CP parameters when 
the fractions are varied one a up and down from their MC 
estimates, or replaced by the values found for the con- 
tinuum background, or assumed to be zero. Similarly, 
the uncertainties due to our knowledge of the right-sign 
fractions for -D° — > K2.K^K~ continuum events and BB 
events are evaluated from the maximum variations of the 
CP parameters after varying these fractions according to 
their MC values or assuming that the is randomly 
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TABLE V: Summary of the main contributions to the Dalitz model systematic error on the CP parameters. 



Source 


X- 


y- 


x+ 


y+ 


x*_ 


y- 


x*+ 


y*+ 


Xg — 


Vs- 


Xs+ 


ys+ 


Mass and width of Breit-Wigner's 


0.001 


0.001 


0.001 


0.002 


0.001 


0.002 


0.001 


0.003 


0.003 


0.001 


0.002 


0.002 


TTTT S-wave K-matrix solutions 


0.003 


0.012 


0.003 


0.001 


0.003 


0.007 


0.002 


0.009 


0.001 


0.001 


0.013 


0.003 


Kn S-wavo parameterization 


0.001 


0.001 


0.002 


0.004 


0.001 


0.003 


0.001 


0.003 


0.005 


0.001 


0.004 


0.002 


Angular dependence 


0.001 


0.001 


0.001 


0.001 


0.001 


0.001 


0.002 


0.001 


0.003 


0.001 


0.003 


0.001 


Blatt-Weisskopf radius 


0.001 


0.001 


0.001 


0.001 


0.001 


0.001 


0.001 


0.001 


0.002 


0.001 


0.001 


0.003 


Add/remove resonances 


0.001 


0.001 


0.001 


0.001 


0.001 


0.002 


0.001 


0.002 


0.001 


0.001 


0.001 


0.002 


Dalitz plot efficiency 


0.006 


0.004 


0.008 


0.001 


0.002 


0.004 


0.002 


0.003 


0.008 


0.001 


0.008 


0.004 


Background Dalitz plot shape 


0.003 


0.002 


0.004 


0.001 


0.001 


0.001 


0.001 


0.001 


0.004 


0.001 


0.004 


0.002 


Normalization and binning 


0.001 


0.001 


0.001 


0.002 


0.001 


0.001 


0.001 


0.002 


0.002 


0.001 


0.003 


0.001 


Mistag rate 


0.008 


0.006 


0.006 


0.005 


0.002 


0.001 


0.002 


0.003 


0.008 


0.010 


0.004 


0.007 


Dalitz plot complex amplitudes 


0.002 


0.002 


0.003 


0.004 


0.001 


0.001 


0.002 


0.006 


0.003 


0.003 


0.004 


0.002 


Total Dalitz model 


0.011 


0.015 


0.011 


0.008 


0.004 


0.010 


0.005 


0.012 


0.014 


0.011 


0.018 


0.010 



TABLE VI: Summary of the main contributions to the experimental systematic error on the CP parameters. 



Source 


X- 


y- 


x+ 


y+ 


x*_ 


y- 


xl 


y*+ 


Xg — 


ys- 


Xs+ 


ys+ 


rriES, AiJ, T shapes 


0.001 


0.001 


0.001 


0.002 


0.002 


0.004 


0.004 


0.005 


0.003 


0.002 


0.001 


0.004 


Real -D° fractions 


0.001 


0.001 


0.001 


0.001 


0.001 


0.001 


0.004 


0.001 


0.002 


0.004 


0.001 


0.001 


Charge-flavor correlation 


0.002 


0.002 


0.001 


0.001 


0.002 


0.002 


0.002 


0.001 


0.001 


0.002 


0.001 


0.001 


Efficiency in the Dalitz plot 


0.002 


0.002 


0.002 


0.002 


0.001 


0.001 


0.001 


0.001 


0.002 


0.003 


0.001 


0.005 


Background Dalitz plot shape 


0.012 


0.007 


0.013 


0.003 


0.010 


0.007 


0.007 


0.007 


0.014 


0.006 


0.012 


0.005 


B- D*°K- cross-feed 










0.003 


0.002 


0.007 


0.001 










CP violation in Di: and BB bkg 


0.001 


0.001 


0.001 


0.001 


0.005 


0.001 


0.001 


0.004 


0.006 


0.002 


0.003 


0.001 


Non-T^"* B- ^ D°K°s7V- decays 


















0.035 


0.058 


0.025 


0.045 


Total experimental 


0.015 


0.007 


0.014 


0.006 


0.014 


0.009 


0.014 


0.010 


0.039 


0.058 


0.028 


0.051 



associated either with a negatively- or positively-charged 
kaon (absence of correlation). 

The effect due to reconstruction efficiency variations of 
the signal across the Dalitz plane, e(m), has been evalu- 
ated by varying randomly the cocfhcients of the polyno- 
mial parameterization according to their covariance ma- 
trix, including the statistical errors due to the limited MC 
statistics as well as systematic uncertainties arising from 
the imperfections of the detector simulation, as discussed 
in Sec. HIE 2. 

The uncertainty associated with the knowledge of the 
Dalitz plot distributions of continuum background events 
is taken to be the difference in the CP parameters us- 
ing background Dalitz plot shapes from sideband data 
instead of signal region backgrounds from MC. We also 
account for statistical uncertainties adding in quadrature 
the r.m.s. of the distributions of CP parameters when the 
two sets of profile distributions are randomized. Uncer- 
tainties due to the Dalitz plot shapes of combinatorial D 
mesons in BB background are conservatively estimated 
from the variation of CP parameters when the reference 
shapes are replaced by a flat profile. 

The effect of the remaining cross-feed of B~ — > 
D*"[£'"7r"]/s:- events into the ^ b*^[b'^-i]K- sam- 
ple (5% of the signal yield) has been evaluated by includ- 
ing in the CP fit an additional background component to 
the latter sample with Vd'^j) identical to that of the 
signal component of the former. 



Possible CP-violating effects in the background have 
been evaluated by setting the CP parameters of the 
B~ — > D''*^^'it~ background component to the values 
obtained from a CP fit to the B~ — * £)(*)°7r~ control 
samples, and by floating an independent set of CP pa- 
rameters for the mixture of BB background. 

The D^K*~ sample has two additional sources 

of uncertainty. The first one comes from signal events 
where the prompt K*~ is replaced by a combinatorial 
K*~ (about 9% of the signal), with either the same or 
opposite charge. This systematic uncertainty, evaluated 
by c;hanging by ±10% the fraction of these events and 
neglecting the charge-flavor correlation, has been found 
to be negligible. 

The second additional uncertainty is due to our knowl- 
edge of the parameter defined in Eq. (3), which ac- 
counts for the interference between B~ D^K*~ and 
other B~ D^K^'it~ (higher K* resonances plus non- 
resonant) decays. Since this parameter cannot be ex- 
tracted from the CP fit and no experimental data analysis 
is available on the B~ — * D'^K^tt^ decay, wc study a B~ 
Dalitz (isobar) model including K*{892)-, is:^(1410)-, 
i('2*(1430)-, i:»*(2010)", 1)1(2460)- and non-resonant 
terms, and randomly varying phases in the range [0, 2tt] 
and magnitudes [50] . The magnitude of the contribution 
from b ^ c transitions relative to 6 — > m was fixed to 
be aroimd 3, while the magnitude of the non- resonant 
contribution was varied between and 1. Since our 
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model has a large uncertainty wc made several alterna- 
tive models adding/removing resonances and changing 
ranges for 6 — > m amplitudes, keeping the K* pollution 
(defined as the non-JsT* fit fraction) below 5-10%, since 
from earlier studies with very similar selection criteria 
we estimate that, neglecting higher resonances, the non- 
resonant K* decays contribute about 5% of the signal 
events [51]. Evaluating k from Eq. (3) for the region 
within 55 MeV/c^ of the K* mass and \ cos6h\ > 0.35 
we find quite narrow distributions, centered around 0.9 
and with r.m.s. not larger than 0.1, in agreement with 
previous studies [52]. For this reason we have fixed the 
value of K to 0.9 in the reference CP fit, and varied it 
between 0.8 and 1. 



V. INTERPRETATION OF RESULTS 

A frequentist procedure [21] has been adopted 
to transform the measurement of the CP parame- 
ters = (x^, y^, x^, yjip, Xszp, yszp) into the mea- 
surement of the physically relevant quantities p = 

Using a large number of pseudo-experiments with 
probability density functions and parameters as obtained 
from the fit to the data but with different values of 
the CP parameters, we construct a multivariate Gaus- 
sian likelihood function £(p[z, C) relating the experimen- 
tally measured obscrvablcs z = {z+,z_} (reported in 
Table IV) and their 12 x 12 statistical and systematic 
covariance matrices C with the corresponding true values 
calculated using their definition in terms of the quanti- 
ties p. The matrices C are constructed from the uncer- 
tainties summarized in Table IV and the statistical and 
systematic correlation coefficients given in Sec. IV A and 
Appendix A, respectively. For a single B decay channel 
the procedure is identical to that outlined here but with 
a reduced space of measured and truth parameters. For 
example, for B~ — > D^K~, z± = {x±,y±), C is the cor- 
responding 4x4 covariance matrix, and p = (7, tb, 5b)- 

We evaluate the confidence level (CL) as a function of 
the true value for a given parameter fi from p = {/i, q}, 
minimizing the function x^(p[z,C) = — 21n£(p[z,C) 
with respect to the parameters q. For each given value 
jjbo of fj,, between its minimum and maximum value, the 
fit provides a minimum chi-square X^(/iOjqo), where qo 
are the best parameters for the given (Iq and the actual 
z measurements with covariance matrix C. Then we take 
the values pbest = {^best, qbost} for which X^(Mbest, qbest) 
is minimum and compute the x^-difference Ax^(a'o) = 

X^(MO,qo) - X^(Mbost, qbest) 

In a purely Gaussian situation for the truth parameters 
p, the CL can be obtained by computing the probability 
that this value is exceeded for a x^-distribution with one 
degree of freedom, CL = 1 — a = i^(Ax^(/Uo); = 1), 
where F{^x^{iJbo);v = 1) is the corresponding cumu- 
lative distribution function. In a non-Gaussian situa- 
tion one has to consider Ax^{no) as a test statistic. 



and has to rely on a Monte Carlo simulation to ob- 
tain its expected distribution. This Monte Carlo sim- 
ulation is built by generating a large number of sam- 
ples with truth values po = {/Uoi qo} as determined from 
the actual data analysis, and then counting the number 
of experiments for which Ax'^(/io) < ^X^ifMt)- where 
Ax'^(/^o) = x'^(Mo,qo) - x'^(Mbest' ^best) is determined 
by letting the q parameters free to vary for each of the 
generated (primed) samples. The one- (two-) standard 
deviation region of the CP parameters is defined as the 
set of Ho values for which a is greater than 31.7% (4.6%). 

This technique to obtain the physical parameters 
takes into account unphysical regions of the parameter 
space [53], which may arise since in the z measurements 
we allow B~ and i?+ events to have different r^- and 
rB+ values, while the space of true values is built using 
a common rs parameter. Moreover, this approach pro- 
vides 1-dimensional intervals that include the true value 
as implied by the confidence level, while in previous mea- 
surements [18, 54] the l-dirnc!nsional intervals were de- 
termined from projections of the multidimensional confi- 
dence regions onto each of the parameters. 

Figure 12 shows a = 1 — CL cLS cL function of the pa- 
rameter 7, for each of the three B decay channels sepa- 
rately and their combination. As expected from Eq. (2), 
the method has a two-fold ambiguity in the weak and 
strong phases, (7; S''*\Ss) ^ {j + 180°; 4*' + 180°, (5^ + 
180°). For the combination of all decay modes wc ob- 
tain 7 = (76^24)° {5,5}° (mod 180°), where the error 
includes statistical, experimental and Dalitz model sys- 
tematic uncertainties. The values inside brackets indi- 
cate the parabolic contributions to the total error com- 
ing from experimental and Dalitz model systematic un- 
certainties. The corresponding two-standard-deviation 
interval is [29, 122]°. The central value is taken at the 
point of maximum a, including all sources of uncertain- 
ties. Considering only D° K^Tr^TT~ samples we obtain 
7= (63t^°)° {8,7}° [5,125]° (mod 180°). 

Similarly, Fig. 13 shows a as a function of the ampli- 
tude ratios rs, r^, and nrs, and the strong phases 5b, 
5*B, and 5^. We obtain rs = 0.086 ± 0.035 {0.010, 0.011}, 
r*s = 0.135 ± 0.051 {0.011,0.005}, Kr^ = 0.163;^:?^^ 
{0.037,0.021} (r, = 0.18i;o j?°), 5b = {WQIlIIY 

{4,7}°, 5*s = {-QStfoY {5,4}°, and 5^ = (104^1?)° 
{17,5}°. The results of the strong phases correspond 
to the solution for 7 in the sheet [0,180]°. The cor- 
responding two-standard-deviation intervals arc rs < 
0.157, r*B G [0.011,0.237], nr^ < 0.338 (r^ < 0.377), 
Sb e [40,166]°, and 6% e [-125,-9]°. No constraint on 
6s is achieved at the two-standard deviation level. 

The significance of direct CP violation is obtained by 
evaluating the CL = 1 — a for the most probable CP con- 
serving point, i.e. the set of coordinates of p with mini- 
mum CL and 7 = 0. Including statistical and systematic 
uncertainties, we obtain CL = 0.971,0.989 and 0.871, 
corresponding to 2.2, 2.5, and 1.5 standard deviations, 
for i)°K-, B- D*°K-, and D°K*- de- 
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FIG. 12: (color online), a — 1 — CL as a lunction ol 7 for 
B- ^ D°K-, B- D*°K^, and B" D^K'" decays 
separately, and their combination, including statistical and 
systematic uncertainties and their correlations. The dashed 
(upper) and dotted (lower) horizontal lines correspond to the 
one- and two-standard deviation intervals, respectively. 



cays, respectively. For the combined analysis of the three 
charged B DK decay modes we obtain CL — 0.997, 
corresponding to 3.0 standard deviations. 



VI. 



CONCLUSION 



In summary, using 383 million BB decays recorded by 
the BABAR detector, we have performed a new measure- 
ment of the direct CP-violating parameters (a;-;^ ) 
and {xsT,ysT) in -> D'^*^^K- and B- D^K*- 
decays, respectively, using a Dalitz plot analysis of D'^ 



and 13° — > K^K^K . Compared to our pre- 



vious analysis based on 227 million BB decays [18], 
this measurement takes advantage of significant improve- 
ments in reconstruction efficiencies, treatment of e+e^ — ^ 
qq, q = u,d,s,c background, and Dalitz models, along 
with the use, for the first time, of D° K'^K^K~ de- 
cays. These upgrades result in reduced experimental and 
Dalitz model systematic uncertainties, and statistical un- 
certainties improved beyond the increase in data sample 
size. The results, summarized in Table IV are consistent 
with, and improve significantly, the previous measure- 
ments from BaBAR and Belle [18, 54]. 

A significant reduction in Dalitz model systematic un- 
certainties has been achieved through the detailed study 
of high-statistics samples of e+e^ cc ^ D^tt^ 
decays, where the D° is reconstructed in the iiTgTr+Tr" 
and K2;K^K~ final states. We have adopted a K-matrix 
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FIG. 13: a = 1 — CL as a function of (a) rs, r'%, and nrs, and 
(b) 5s, 5*B, and 5,, for B' D^K", B~ -» D*°K-, and 
B~ — > K*~ decays, including statistical and systematic 
uncertainties and their correlations. The dashed (upper) and 
dotted (lower) horizontal lines correspond to the one- and 
two-standard deviation intervals, respectively. 



formalism to describe the complex tttt and Kn S-wave 
dynamics in _D" K^Ti^n^. For this decay, the fit 
fractions measured and reported in Table II, show that 
the Kit and tttt P-waves dominate, but for the first time 
significant contributions from the corresponding S-waves 
are observed (above 6 and 4 standard deviations, respec- 
tively). 



Using a frequentist analysis we interpret the 



S*) 



(*) 



and (xgip, Hs^) experimental results in terms of the weak 
phase 7, the amplitude ratios rg, r^, and Ts, and the 
strong phases 5b, 5*^, and 5s- We obtain 7 — (76 ± 22 ± 
5±5)° (mod 180°), where the first error is statistical, the 
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second is the experimental systematic uncertainty and 
the third reflects the uncertainty on the D decay Dahtz 
models (parabolic errors). The corresponding two stan- 
dard deviation region is 29° < 7 < 122°. The combined 
significance of direct CP violation (i.e. 7 7^ 0) is 99.7%, 
corresponding to 3.0 standard deviations. This direct 
determination of 7 supersedes and significantly improves 
our previous constraint [18], and is consistent with that 
reported by the Belle Collaboration [54]. The latter has 
a slightly better precision in spite of a larger uncertainty 
on the measured CP parameters because the error on 7 
scales roughly as l/r^-'(l/rs) and our tighter r^^\ Krg 
constraints favor smaller values. 
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APPENDIX A: CORRELATION MATRICES 

Averaging .x^\ .Xg^, jy,,^ measurements be- 

tween different methods and experiments, and tran- 
forming into the physically relevant quantities p = 
(7, r^, 7"^, urg, SB,Sg,Sg) requires a complete evaluation 
of the different sources of uncertainties and their cor- 
relations. The statistical correlation coefhcients are ex- 
tracted from the fit and are reported in Sec. IV A. The 
experimental systematic and Dalitz model systematic 
correlation coefiicients for the measurement vector 

z = {x-,y-,x+,y+,x*_,y*_,x*^_,y*+,Xs-,ys-,Xs+,ys+), 

are defined in the usual way as pij = Cij/ ^yCuCjj, where 

Cij = (z - Zbest)i(z - Zbest)j, with Zbest the vector of 

best measurements, and equal to (only diagonal and 
lower off-diagonal terms are written, in %): 
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